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Abstract — Generalized Orthogonal Matching Pursuit 
(gOMP) is a natural extension of OMP algorithm where 
unlike OMP, it may select N(> 1) atoms in each iteration. 
In this paper, we demonstrate that gOMP can successfully 
reconstruct a if -sparse signal from a compressed measure- 
ment y = <l?x by if" 1 iteration if the sensing matrix 4? 
satisfies restricted isometry property (RIP) of order NK 
where Snk < "T^ffy^- O ur bound offers an improvement 
over the very recent result shown in |1|. Moreover, we 
present another bound for gOMP of order NK + 1 with 
Snk+i < ^rpf^^N which exactly relates to the near optimal 
bound of S K +i < -7=— j- for OMP (N = 1) as shown in 

I. Introduction 

Compressed sensing or compressive sampling (CS) 
|3|-[5| is a powerful technique to represent signals at 
a sub-Nyquist sampling rate while retaining the capacity 
of perfect (or near perfect) reconstruction of the signal, 
provided the signal is known to be sparse in some 
domain. In last few years, the CS technique has attracted 
considerable attention from across a wide array of fields 
like applied mathematics, statistics, and engineering, 
including signal processing areas such as MR imaging, 
speech processing, analog to digital conversion etc. Let 
a real valued, band-limited signal be sampled following 
Nyquist sampling rate over a finite observation interval, 
generating anxl signal vector u = (ut, u%, ■ ■ ■ , «„) T '. 
The vector u is known to be if -sparse under some 
transform domain 

u = *x 

where f is 11 x n transform matrix and x is the cor- 
responding n dimensional transform coefficient vector 
that is approximated with at most if non-zero entries. 
Suppose that the signal u is converted to a lower 
dimension (m) via linear random projection 

y = Au 
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where y is observation vector with m <C n and A 
is a flat to x n random matrix. According to the CS 
theory, it is then possible to reconstruct the signal u 
exactly from a very limited number of measurements 
M = 0(K\og e n). Therefore, CS framework results in a 
potential challenge in reconstructing a If -sparse signal 
from a under determined system equation 

y = *x 

where <& = A\& is a to x n dimensional sensing matrix. 

Under the if -sparse assumption x can be recon- 
structed by solving the following Iq minimization prob- 
lem 

min ||x|| subject to y = <&x. (1) 

[Note that uniqueness of the if -sparse solution requires 
every 2if column of to be linearly independent.] The 
above Iq minimization problem provides the sparsest 
solution for x. However, the Iq minimization problem 
is a non-convex problem and is NP-hard. The feasible 
practical algorithm for this inverse problem may be 
broadly classified into two categories, namely convex 
relaxation and greedy pursuits. 

1) Convex Relaxation: This approach translates the 
non-convex Iq problem into relaxed convex problem us- 
ing its closest convex Zi norm. This imposes "Restricted 
Isometry Property (RIP)" condition of appropriate order 
on <I? as defined below. 

Definition 1. A matrix $ mxn satisfies RIP of order K 
if there exists a constant 8 G (0,1) for all index set 
I C {1, 2, • • ■ , N} with \I\ < K such that 

(l-«)||q||l<||*/q||l<(l + *)||q||l. (2) 

The RIP constant 8k is defined as the smallest value of 
all 8 for which the RIP is satisfied. 

There are three main directions under this category, 
namely the basis pursuit (BP) [6], the basis pursuit de- 



noising (BPDN) |7] and the LASSO ©. The reconstruc- 
tion problem is formulated under them as, 
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1. BP: 



mm 



jxj^ subject to y = <l?x 



BPDN: 



3. LASSO: 



min A 
min 1 1 



s.f r 



^ e 



The BP problem can be solved by standard polynomial 
time algorithms of linear programming. The exact K- 
sparse signal reconstruction by BP algorithm based on 
RIP was first investigated in [9] with the following bound 
on 5 : 5k + 5 2 k + $3K < 1- Later the bound was refined 
as S 2K < V2-1 HO), 6i.tbk < \/2 - 1 CD and 
5 2K < 0.4652 Ull. The BPDN and LASSO problem can 
be solved by efficient quadratic programming (QP) like 
primal-dual interior method. However, the regularization 
parameters A and e play a crucial role in the performance 
of these algorithms. The convex relaxation technique 
provides uniform guarantee for sparse recovery. How- 
ever, the complexity of l\ minimization technique is 
large enough (C(n 3 )) for some applications (e.g. real 
time video processing). 

2) Greedy Pursuits: This approach recovers the K- 
sparse signal by iteratively constructing the support set of 
the sparse signal (i.e. index of non-zero elements in the 
sparse vector). At each iteration, it updates its support set 
by appending the index of one or more columns (called 
atoms) of the matrix (often called dictionary) by some 
greedy principles. This category includes algorithms like 
orthogonal matching pursuit (OMP) [13], generalized or- 
thogonal matching pursuit (gOMP) [1|, |14|, orthogonal 
least square (OLS) lfl"5l .compressive sampling matching 
pursuit (CoSaMP) £16), subspace pursuit (SP) H7) and 
so on. These algorithms offer very fast convergence 
rate with high accuracy in reconstruction performance, 
but they lack proper theoretical convergence guaranty. 
Among these greedy algorithms, OMP is widely used 
because of its simplicity. The theoretical guaranty of 
OMP algorithm for an exact recovery of the sparse 
signal under a K + 1 th order RIP condition on 3? is 
improved in the following way: 8k+i < in lfl8l l 



>K+1 



< 



in 



5k+i < 



1+2-/K 

1 in m 
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A'+l 



< 
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3VK . 

in 



[14| and 



A. Our contribution in this paper 

In this paper, we have analyzed the theoretical perfor- 
mance of gOMP algorithm in a different approach and 
our theoretical result improves the bound on RIP of order 
NK from 5 NK < [1] to 5 NK < ^M^. 

we have also presented another approach which results in 



a RIP bound of order NK + 1 with 8nk+i < —= — — • 

VK+VN 

Finally, we have discussed the theoretical performance 
of this algorithm under noisy measurement and proposed 



a bound on signal to noise ratio (SNR= 



<j> x reconstruction of support set. 
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) for correct 



B. Organization of the paper 

Rest of the paper is organized as follows. Next section 
presents the notations used in this paper and a brief 
review of OMP and gOMP algorithms. In section III, 
theoretical analysis of gOMP algorithm for noiseless 
observations is presented. In section IV, analysis of this 
algorithm in presence of noise is provided. Discussion 
is presented in section V and conclusions are drawn in 
section VI. 

II. Notations and a brief review of OMP and 
gOMP algorithms 

A. Notations 

The following natations will be used in this paper. Let 
the columns of matrix be called as atoms where $ = 
[0i</>2</>3- ■■4>n\- The matrix &a represents the sub-matrix 
of $ with columns indexed by the elements present in 
set A. Similarly x^ represents the sub-vector of x with 
elements whose indices are given in set A. T is the true 
support set of x and A k is the estimated support set after 



k iterations of algorithm. $ 



t 

A 1 ' 



A*, 



A*' 



the pseudo-inverse of Qi^k. Here we assume that 
has full column rank (A k < m). P^k = &Ak<f>\ k is 
the projection operator onto column space of <J? A k and 
P^ fc = I P A t is the projection operator upon the 
rejection space of span(A' c ). A A i- = P^>3? is a matrix 
obtained by orthogonalizing (projecting onto rejection 
space) the columns of <fr against span(<& A fc). 
For referring to previous results we use the following 
notation. Suppose an equation follows from the result 
of Lemma 1 then LI is mentioned at the top of the in- 

Ll 

equality /equality like > . Similarly if an equation follows 
from another equation or defination or theorem then it 

is mentioned as = or = X or > respectively. 



B. A brief review of OMP and gOMP algorithms 

The algorithm is presented in Table 1. The OMP 
algorithm starts with an empty support set A and 
keep selecting a single atom in every iteration based 
on highest correlation with residual signal r fc_1 until 
the support set is full with the index of K atoms. At 
k th iteration, the residual signal r k is updated using the 
difference between signal y and its orthogonal projection 
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TABLE I 
OMP ALGORITHM 



Input: measurement y£ R m , sensing matrix«I> mxn 
Initialization: counter k=0, residue r°=y, 
estimated support set A k = 
While k<K and ||r fc || 2 < ||r fc - 1 |j 2 
k=k+l 

Identification: h k =arg maxj \(r k ~ ,&j)\ 
Augment: A k = A fe ~ 1 U {h k } 

Estimate: x Afc =arg mm z . supp{z)eAk \\y - # Afc z|| 2 
Update: r k = y — * At x Ai 
End While 

Output: x =arg mm z:supp(z)eA K \ \y - $ A kz\\ 2 

In gOMP algorithm the Identification step is 

only different. We select a vector h* € M. N 

which has N largest entries in \& T r k ~ 1 \.(N K < m) 



on the subspace spanned by the atoms corresponding to 
the current support set A k . Generalized OMP algorithm 
is very similar to OMP where N largest correlated 
atoms are selected in each step. This simple modification 
in identification step results in improved reconstruction 
performance for A"-sparse signal 0J. 



III. Analysis of gOMP 

To analyse gOMP algorithm we use some commonly 
used properties of RIP as summarized in Lemma 1. 

Lemma 1. (Lemma 1 in KTTi Kffli ) 

a) &Kx < 8k 2 VKi < A 2 (monotonicity) 

b) (1 - <5 m )||q|| 2 < ||*f*/q|| 2 < (1 + <V|)||q|| 2 

c) ||*fq|| 2 < v/1 + <5|/|||q||2 

d) ($/q, *jp) < ||q|| 2 ||*7*jp||2 < 

^i/i+|j|l|p||2||q||2 

for I,JC {1, 2, n}, \I\, q£l' and p € R' 7 

Note that, the algorithm can reconstruct a A-sparse 
signal by K th iterations if atleast one correct index is 
chosen in each iteration. Now, let in k + 1 th iteration 







(<p,r k ) for i e T and oq 



for j 



satisfies RIP of order NK with 



Snk < 



N 



Proof. To start with we use the same upper bound on 
a N as presented in (T). Interested readers may refer fll] 
for proof of the following lemma. 

Lemma 2. (Lemma 3.6 in 

k ^ Snk ||x T _ A fc|| 2 

a N < 



1 - Snk 



N 



Now we go about finding a better bound on j3 k . We 



observe that r fe 

any i <E A k 

„fc\ 



P-h,y and /3 k = ||€>yr fe| 



and for 



i! P^r fe > = <P^^,r fe >=0 (3) 



So, 



1 



||*^r fc |U > — ||$^r fc || 2 (as \T\ = A) 
v A 

(3) 1 u,, 1 



T—A k 



=r 2 = 



A 



■ T—A k A 



A 



I* 



(p^fp^ylh (as p = P J &P = P*) 



T-A' 



1 



IK 

(3) 1 



A 



(P^* T _ Afc ) T Pi fc * T x T || 2 

I (Pa* *T-A* ) T Pi> *&T—A k X T— A fe 1 12 



(4) 



Now, to proceed further we require the following lemma. 

Lemma 3. (Extension of lemma 3.2 from [18]) 
A™ x ™ satisfies modified RIP of Jl^l±iM 



(1- 



s 



1-6 
and also 



\h\+\h\ 
6 \h\+\h\ 



\u\\l < \\A h u\\l < (l+<Vi|+|/2|)ll u 



1 - d |/i|+|/ 2 | 

where u€ W 1 , A,/ 2 € {1, ...,n) supp(u) e 7 2 and 
h n J 2 = 



Proof: In Appendix A 



P^*x' = P| fc [* T _ 



A* 



T where /3f 's and a k 's are arranged in descending Now Let x' = 
order. So (3 k > > /3 N are N largest correlations in 
support set and similarly a k > a k ... > a N are N largest 
correlations of incorrect indices. Now if we ensure that 
j3 k > a N then atleast f3 k will appear in the overall N 
largest correlated atoms which are selected. Hence, we 
find the lower bound of f3 k and upper bound of a N and 
compare them. In this paper, we propose two RIP bounds 
which are presented as Theorem 1 and Theorem 2. 

Theorem 1. gOMP can recover x exactly when $ 



^T-A k 





x' e 



-T-A k V (T-A k y 



So, A A fcx' = 



X T _ A fc 





P| fc *T-A fcX T-A fc and also P{ fc *TXT 



Pi>[*T-A» * (TnAfe) ] 

L x TnA fc 

P^ fc $7-_ A fcX T _ A fc. Hence we get 

A Afc x' = P^*x' = P^Syxr 

= ^ = P A**T-A* X T-A* 



(3) 



(5) 
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Hence, from Lemma 3, with I\ — A k and I2 = 



supp(x') = T - A fc and h n h = 1 
Nk + K - I where T n A fc = I we get 



I/, 



1 v-^'lla > C 1 



I A A fcx' 



Lla , 
> (1- 



>Nk+K-l ■ 
- &Nk+K-l ' 



I X T-A fc ll2 



,'l 1 2 



/ 2 = 




h\ = 










A* 1- A* 


(6) 


iJ , nA t i=/ 



Moreover, 

II A 'l|2 ( 5 ) 

l|A A fcX || 2 = 



I Pa* *r-A" x t-a<= 1 1 2 



(Pi**T-A*Xr-Afc 1 PA**r-A* x T- 
(( P A* *T-A* ) T -PAfc *T-A* X T-A* . X T- 



Fig. 1 . Venn diagram for con'ect and estimated support set 

where (10) follows from the fact that 

P A fc«& T _ A fcX T _ A fc € span(<& A k) and it can 

be viewed as <fr A kZ A k where z A t 6 M A and 



< 



A* 



i?T-A k ) - r A fc T-A fcX T-A fc ! x T-A fc / De vieweu as v A fcZ A fc wnere z A t t jk anu 

*r-A*) T -PA**r-A*»r-A*ll2||»r-A*ll2 ZAfc = *L*t-a* x t-a*- So x^ uAfc is a vector in 



(7) 



Combining (6) and (7) we get 

1 1 (Pf » * T _ A , ) T P& * T _ A , x T _ A , I | a 

> (1 - 1 c )II X T-A* 2 

Therefore combining this result with (4) and (8) we get 



(8) 



Pi > 



1 ,j £>a»"a: « 
AT I-^ata-' 



| X T-Afc 2 



(9) 



Making lower bound on /3J greater than upper bound of 
(from Lemma 2) bring us to the result. □ 



The next theorem states the other bound for gOMP 
success. 

Theorem 2. gOMP can recover x exactly when 4? 
satisfies RIP of order NK + 1 with 



$NK+1 < 



N 



K 



N 



Proof. Let us begin by examining the residue r k in the 
fc+1*' 1 iteration. In (2) it was shown that r fc E span^x) 
for OMP where estimated support set A k C T. Now 
we show that in cases where A k and T are in general 
modelled as shown in Fig.l, r fe is indeed spanned by 
* TuA k. 

r k = y-* Afc * Afc y 
= * T x T - P Afc y 

= *T-A fcX T-A fc + *TnA fcX TnA fc 

- P A fc(* T _ A fcX T _ A fc + <fr TnA fcX TnA k) 

= *T-A<= X T-A<= - P A fc*T-A fcX T-A fc 

= * T _ A /cX T _ A fc - * A fcZ A fc (10) 

= *TUA fcX TuA fc (11) 



(12) 



A . Observe that 

„ _ XT-A k 
x TUA fc — _ , 

Let W be the set of remaining incorrect indices over 
which a k 's are chosen (W e (TU A fc ) c ). So, 

a k N = min((®i,r k )) (i £ W) 



< 



N < 
|$^r fc || 2 



J2 a 

N 



(as \W\ = N) 



N 



(11) 1 



N 



l*W'*TUA fcX TuAfcll2 



Lid 



>N+Nk+K-l I \ ^TUA k 1 12 



Lla 1 JC II " II 

< -^ d A'A'+l|| x TuA fc ll2 



(13) 



where (13) comes from the fact that / > k and k < K—l. 
Now for finding lower bound of [3 k in terms of 

ll x TuA fc H 2 we P rocee d m miS wav - 

p k = H^iu 

>^||*?r fe || 2 (as\T\=K) 



(14) 



K 

— 1| [$ T * A A,_ T ] T r fc || 2 
V iv 



— "y^ll*TuA fc *TuA fcX TuA fc H2 

> -^=(1 - (5jVfe + A'-i)ll x TUA*l(2 

> -7= (1 - OArA-)||x TUA >||2 



(15) 



where (14) comes as 3> Afc _ T r fc = which follows from 
(3). Now from (13) and (15) ensuring /3 k > gives 
us the result. □ 
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IV. Analysis in presence of noise 

In case of noise we can model the measurement as 
y' = y + n = $x + n where n is the added noise. We 
can show the performance of this algorithm in presence 
of noise in two ways. One is by finding the upper bound 
of reconstruction error energy ||x— x^t H2 (as presented 
in [fl~))and other is by providing a condition for exact 
reconstruction subject to upper bound on measurement 



SNR= 



.llylh 



Theorem 3. If k = K forms the stopping criterion 
in gOMP with 3? satisfying S^k+k < 1 an d Snk < 
then ||x — Xa*:||2 < Ci<"i 1 1 n| I2 where 



v/lv 



VK+2VN 



c 



)(t/N(1+6 k ) + v /K(1+6 n )(1+5 k )) 
{VN-(^+2VN)S NK )y/l-8 NK+K 



yl— &N K+K 

Proof: In Appendix B 

Theorem 4. If k — K forms the stopping criterion in 
gOMP with satisfying Snk+k < 1 and 5nk+i < 



then 



- x A k || 2 < Cif 2 1 |n|| 2 where 

(^N(1+8k)+^ K(1+S n )(1+5k)) 



(Vn-(Vk+Vn)S nk+1 )^/i-s„ 



2 

Proof In Appendix C 

The above bounds provide a estimate on upper bound 
on reconstruction energy. But they do not guarantee 
estimation of correct support set. In some cases it may 
happen that the reconstruction error energy is bounded 
but the chosen support set is completely different. This 
may prove expensive because in most cases knowledge 
of correct support set is more important than knowledge 
of exact value at that position. So easily verifiable bounds 
guaranteeing reconstruction of correct support set are 
necessary. 

In communication we often judge the performance by 
the SNR of the received signal. Before applying sparse 
reconstruction we do not have the information of energy 
of vector x. But we do have knowledge of energy of 
clean measurement from transmitter's end. Hence by 
calculating the SNR at the receiver's end we can have an 
idea whether a particular algorithm can be implemented 
for reconstruction or not. This can be a good measure of 
performance analysis for reconstruction algorithms. So 
we present a bound on jj^jj' for which correct support 
set is estimated. Before stating the theorem let us analyse 
the assumption made: |x,;| > ^p-Vi, j £ T. This implies 
that all non zero values of x are bounded within some 
ratio of the maximum. We see that the sparse systems 
are modelled by setting the values of elements in x 



below some threshold as zero. Hence this assumption 
can always be made. If suppose x has a non-zero value 



below ^" ax| then it can be modelled as a K — 1 sparse 
system by setting that value to zero without affecting the 
output much. 

Theorem 5. If measurement y = <&x is corrupted with 
noise n then gOMP algorithm can still recover the true 
support ofx provided TW! 2 > Ck 3 where 



Ck 3 — 



jK{l + 8 N ) + y/N{l + 8 K ) 

^N(1-S K )(1-28 NK ) 



^^(I+SkKISnk) 2 



y/K(l + S N ) 



Proof. At first let us make use of the assumption and 
provide a result which would be used in subsequent proof 



Lemma 4. With usual notations we see that 



|#T-A*X T _A*||2 > 



\K-l){l-5 K ) \\$TXrh 
K(1 + S K ) 7 



proof: In Appendix D 

Let us again compute the bounds on and /3f 



H^P^y'lh 
a N < — 



N 



Lie yTT^HP^y'lh 



N 



< 



< 



vT+Tivlly'lb 



N 



VI + ^7v(||*TXt||2 + IMI2) 

7n 



(16) 



5 



Pi > 



1 



|*?_Afc P A»>(*T-A*X T _A». +n)|| 2 



Lie 1 

> 



*T-A<= P A fc *T-A fc X T-A>= I U 



K 



K 

(7) ||A A *X'|| 2 



n l|2 

J 1 |2 



> 



vtt^ 



y/K — Z||x T _ A k 



K 



n 2 



^3 (l-(Tfe) 2 )II^T-A^ r _ Afc ||l 



\/K — /||x T _ A fc ||2 



K 



n 2 



Dl (1 - 2(5jv g )Vl - fc||^T-A fcX T-A fc l|2 



VK~l(l - 8 NK f 



VF 



n 2 



« (l-2(5 AfA -)(l-^)||# T x T || 2 _ + 

(1 - (WJW^i + <**■) v# 

(17) 

Making < /3f ( (16) < (17) ) for correct choice 
of index we get the desired SNR bound. □ 

V. Discussion 

The proposed bound in Theorem 1 is better than the 
one from [ 1 1 because while obtaining lower bound of 
Pi instead of applying successive inequalities we use a 
more direct inequality presented in Lemma 3 which leads 
us to a higher lower bound. This bound is also better 
than the bound proposed in 03] (Snk < (2+ ^? v ^ ) 
for N < 1A5K. But according to 1 1 1 gOMP performs 
better than OMP for small values of N only. 

It is difficult to compare the bounds presented in 
Theorem 1 and Theorem 2 since Snk < Snk+i and 



VN 



> 



/N 



-= =. But intuitively we can see that 

bound on Snk+i is more optimal since it reduces to near 
optimal bound on OMP for special case of N = 1. The 
proposition on SNR seems to be a good approach since it 
is an easily measurable quantity and can be used in future 
research for comparing greedy algorithm's performance 
under noise. 



VI. Conclusions 

In this paper, we have given an elegent proof of 
the theoretical performance of gOMP algorithm. Our 
analysis improves the bound on RIP of order NK from 



paper, we have presented another bound of order NK+l 



with RIP constant Snk+i < 



We have also 



Vk+VN' 

presented improved theoretical performance of gOMP 
algorithm under noisy measurements. 



Appendix A 
Proof of lemma 3 



We know that 



|su||! = ||p 7l *ui|! 



|P|*u|' 2 



/1 Yu ll2 
2 'I|P/1*U||2 



|<j> u l|2 ||r>_ A„l|2 



(A.l) 



Now (P/^u, *u) = (*u) T Pf i *u 
(4>u) T Pf i P/ 1 $u = llP/j^ulll- Further we see 
that P^ffru e span (3? /J. So P 7l $u = $z for some 
zeE" with supp(z) e h- 



HP/^ul 
||*u|| 2 

Dl,Lld 
< 



(P 7l $u,$u) 
llPi^ullall^ulla 

S \h\+\h\ 



(<I?z, <&u) 
l$z|| 2 ||*u| 



V 1 -^iiw/ 1 - s \h\ 



6 |/l| + |/2| 



Vll+I^l 



(A.2) 



So from (A.l) and (A.2) we getHA/juHl > (1 - 



( 



°\ii\+\h\ 
1 - s \h\+\h\ 



) 2 



l* U ll2 



>(1 _ ( Wi )2)(1 _ 



Lla 

> 



(1 



1- 



Vl\ + \l2\ 



Apply- 



0|/ 2 |j||U|L , c 

1 - d |/l| + |/2| 

ing HP/^uIIj > in (A.l) the upper bound 
becomes ||A /lU || 2 < ||$ u ||| < (1 + 5|j 2 |)||u||| < 

(! +^|/i| + |/2|)ll U ll2- 



Appendix B 
Proof of theorem 3 

First we need to find the bounds on a N and (3^ in 
presence of noise in k+l th iteration 



a k N < 



N 



— ||*r ( p± y + p± n) 



< —(\\^P^y\\ 2 + 11*^11112) 



L2,Llc 1 
< 



< 



Nl- Snk 
1 S NK m 



|X T _ A fc || 2 



VI + Sn . 



N 



P A fcn|| 2 



N 1 



INK 



l x T-A fc l|2 



VI + s N . 



N 



n|| 2 (B.l) 



6 



and 



K > 



l*?_ A ,Pi fc y'|| 2 



> -=ll* 



^P^* T x T ||2 - -f=\\<&T_ A kPl k n\\ 2 



Appendix C 
Proof of theorem 4 

In this case we find the bounds on a k N and fi\ in 
presence of noise similar to our proof on second bound 
of gOMP. Proceding similar to (B.l) and (B.2) we get 



(9),Llc 1 



s 



NK 



i - s NK 



l x T-A fc l|2 



VT+S 
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X T-A fc l|2 T^t ll n l|2 



(B.2) 



Now at the end of algorithm it may happen that some 
incorrect atoms are chosen. Lets say this happens for the 
first time in the p+l th step. Then at this particular step 
(B.1)>(B.2). Which implies 



nk ^ 1 ~ $NK | | ,/ || VI + I, || 

Pi > -y= l|x T uA"ll2 T^ll 11 " 2 

So failure at p+1*' 1 step implies 

y/N(l + 6 K ) + y/K(l + S N ) 

(Vn - (Vk + VN)5 NK+1 ) 



x; 



TuAp 



2 < 



(C2) 



l«l|2 
(C3) 



Now to get an upper bound in estimation error we 
(1 - S NK )(y/N(l + S K ) + y/K(l + S N j) proceed similarly as in (B.4) 



{VN-(VK + 2VN)5 NK ) 
x ||n|| 2 (B.3) 
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The error in reconstruction energy can be seen as 

m ||*x-$x A k|| 2 
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Appendix D 
Proof of Lemma 4 

£>1 , 

||*T-A fcX T-A fc l|2 > V 1 — ^A-||x T _ A fc lb 

> V '(1 - - 0|Xmm| 

'(1 -foXif-Q ||x T || 2 



(C4) 



yl — Snk+k 
Pf P ($ T x T + n)|| 2 



(as ||r 4 || 2 < ||r?|| 2 for i > j) 

n l|2 
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(D.l) 



y 7 ! — (5atk + k 
P^ p * T x T || 2 + ||PA P n|| 2 + ||n|| 2 



yjl — 5 NK+K 

P^ p *t-ApX T _ Ap || 2 + 2 1 1 rx| 1 2 
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4> T -ApX T _ Ap || 2 + 2 1 1 n| 1 2 



sj\ — Snk+k 



Vl + ^||x T _ AP || 2 + 2||n|| 2 
yl — Snk+k 

By using (B.3) in (B.4) we get the desired bound. 



(B.4) 
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